Robust Semantic Segmentation With Multi-Teacher Knowledge Distillation

Recent studies have recently exploited knowledge distillation (KD) technique to address time-consuming annotation task in semantic segmentation, through which one teacher trained on a single dataset could be leveraged for annotating unlabeled data. However, this context, capacity is restricted, and variety rare different conditions, such as cross-model KD, the KD prohibits student model from distilling information using cross-domain context. To fix concern, we looked into learning lightweight group of teachers. more specific, train five distinct convolutional neural networks (CNNs) segmentation several datasets. Several state-of-the-art augmentation transformations also been utilized our training phase. The impacts scenarios are then assessed terms robustness accuracy. As main contribution paper, proposed multi-teacher paradigm endows with ability amalgamate capture illustrations sources. Results demonstrated that method outperforms existing both clean corrupted data while benefiting score weight system. Experiments validate framework results an improvement 9% up 32.18% compared single-teacher paradigm. Moreover, it surpasses previous supervised real-time challenge.